An electronic imaging system typically produces a signal output corresponding to a viewed object by spatially sampling an image of the object in a regular pattern with an array of photosensitive elements, such as, for example, a charge-coupled device (CCD) or Complementary Metal-Oxide Semiconductor (CMOS) solid-state image sensor. In such an imaging system, it is well known that components in the object field that contain fine details can create spatial frequencies too high to be captured in the image without sampling error within the sampling interval of the sensor. These details can produce lower frequency components, resulting in imaging errors commonly referred to as aliasing or undersampling artifacts. Aliasing is related to the system modulation transfer function (MTF) and, in a more pronounced manner, to the spatial periodicity of the photo sites or “pixels” of the solid-state imaging array. In particular, if the spatial detail that is being imaged contains a high frequency component of a periodicity greater than twice the pitch of the photo sites or pixels of the image sensor, the undesirable effect of this high frequency component can be a spurious signal due to aliasing. As is familiar to those skilled in the digital imaging arts, the particular frequency above which aliasing is likely is termed the Nyquist frequency.
In general, the electronic imaging system can reduce aliasing if its optical section has a frequency response that cuts off, or filters out, the higher frequency content of the object being imaged, that is, frequencies above the Nyquist frequency. As a result, the optical section generally employs an optical low pass spatial filter to substantially reduce the high frequency component contained in the spatial detail of the image received by the image sensor. Thus, conventional design of electronic imaging systems involves a trade-off between image sharpness, which increases with higher frequency image content, and compensation for aliasing distortions or undersampling artifacts, which reduces higher frequency image content.
To limit aliasing artifacts, an optical spatial filter, for example, a birefringent anti-aliasing filter (also known as a blur filter), has become a common component in consumer color video cameras. For example, U.S. Pat. Nos. 4,101,929 to Plummer and 4,896,217 to Miyazawa et al. show typical examples of anti-aliasing filters. Such a filter is usually placed between a lens and the image sensor in order to provide a low-pass spatial filter function, reducing the spatial frequency content of the image at frequencies above the Nyquist frequency of the image sensor. This use of an anti-aliasing filter makes the imaging system less susceptible to aliasing distortion. An excellent discussion of aliasing and the use of anti-aliasing filters is presented in U.S. Pat. No. 6,040,857 by Hirsch et al. Another less desirable option to reduce aliasing would be to use a lens with lower MTF at high frequency or a higher f/# lens to blur the image. However, this approach leads to less sharpness in the image or f/# dependent blur and is not a favorable solution for image anti-aliasing.
Recently, image sensors having the ability to image in multiple resolution modes have been commercialized. This innovation in imaging technology allows a single image sensor to have both a high-resolution mode, obtaining a digital image data value from each individual pixel, and one or more lower-resolution modes, in which charge from multiple pixels can be summed together electrically on the image sensor in a process known as binning, thereby reducing the amount of data obtained and effectively obtaining information from fewer, “larger” pixels. Other methods to produce effectively larger pixels include summing pixel values digitally or summing the voltage associated with each pixel and possibly other techniques. In some lower resolution modes such as for the preview images, a sparse sampling of the pixels is used where some of the pixels on the image sensor are not used. Combinations of the various methods are also possible such as a sparse readout of binned pixels that are later summed. Each resolution mode, then, has different sampling characteristics but works with a lens having the same MTF. As the resolution of the image sensor decreases in lower resolution modes, due to increases in the effective size or pitch of the pixels or a decreases in the spatial sampling frequency on the image sensor, the Nyquist frequency goes down and as a result, the tendency for aliasing to occur in an image increases.
Because high-resolution and low-resolution modes require different amounts of optical blur to prevent aliasing and to preserve sharpness, compensating for aliasing with such a dual-mode system can involve a considerable amount of compromise. An anti-aliasing filter that is designed to anti-alias the image in the lowest resolution mode will excessively blur the image in a higher resolution mode. An anti-aliasing filter that is designed for the highest resolution mode will anti-alias properly for high-resolution operation, but will not effectively compensate aliasing for all appropriate frequencies in reduced resolution modes.
Thus, it can be seen that there is a need for solutions that provide anti-aliasing compensation for imaging systems that have multiple resolution modes.